Offline Equilibrium Finding

Offline reinforcement learning (Offline RL) is an emerging field that has recently begun gaining attention across various application domains due to its ability to learn behavior from earlier collected datasets. Using logged data is imperative when further interaction with the environment is expensive (computationally or otherwise), unsafe, or entirely unfeasible. Offline RL proved very successful, paving a path to solving previously intractable real-world problems, and we aim to generalize this paradigm to a multi-agent or multiplayer-game setting. Very little research has been done in this area, as the progress is hindered by the lack of standardized datasets and meaningful benchmarks. In this work, we coin the term offline equilibrium finding (OEF) to describe this area and construct multiple datasets consisting of strategies collected across a wide range of games using several established methods. We also propose a benchmark method -- an amalgamation of a behavior-cloning and a model-based algorithm. Our two model-based algorithms -- OEF-PSRO and OEF-CFR -- are adaptations of the widely-used equilibrium finding algorithms Deep CFR and PSRO in the context of offline learning. In the empirical part, we evaluate the performance of the benchmark algorithms on the constructed datasets. We hope that our efforts may help to accelerate research in large-scale equilibrium finding. Datasets and code are available at https://github.com/SecurityGames/oef

Use of Mobile Phones as Intelligent Sensors for Sound Input Analysis and Sleep State Detection

Sleep is not just a passive process, but rather a highly dynamic process that is terminated by waking up. Throughout the night a specific number of sleep stages that are repeatedly changing in various periods of time take place. These specific time intervals and specific sleep stages are very important for the wake up event. It is far more difficult to wake up during the deep NREM (2–4) stage of sleep because the rest of the body is still sleeping. On the other hand if we wake up during the mild (REM, NREM1) sleep stage it is a much more pleasant experience for us and for our bodies. This problem led the authors to undertake this study and develop a Windows Mobile-based device application called wakeNsmile. The wakeNsmile application records and monitors the sleep stages for specific amounts of time before a desired alarm time set by users. It uses a built-in microphone and determines the optimal time to wake the user up. Hence, if the user sets an alarm in wakeNsmile to 7:00 and wakeNsmile detects that a more appropriate time to wake up (REM stage) is at 6:50, the alarm will start at 6:50. The current availability and low price of mobile devices is yet another reason to use and develop such an application that will hopefully help someone to wakeNsmile in the morning. So far, the wakeNsmile application has been tested on four individuals introduced in the final section

Emerging stability of forest productivity by mixing two species buffers temperature destabilizing effect

The increasing disturbances in monocultures around the world are testimony to their instability under global change. Many studies have claimed that temporal stability of productivity increases with species richness, although the ecological fundamentals have mainly been investigated through diversity experiments. To adequately manage forest ecosystems, it is necessary to have a comprehensive understanding of the effect of mixing species on the temporal stability of productivity and the way in which it is influenced by climate conditions across large geographical areas. Here, we used a unique dataset of 261 stands combining pure and two-species mixtures of four relevant tree species over a wide range of climate conditions in Europe to examine the effect of species mixing on the level and temporal stability of productivity. Structural equation modelling was employed to further explore the direct and indirect influence of climate, overyielding, species asynchrony and additive effect (i.e. temporal stability expected from the species growth in monospecific stands) on temporal stability in mixed forests. We showed that by adding only one tree species to monocultures, the level (overyielding: +6%) and stability (temporal stability: +12%) of stand growth increased significantly. We identified the key effect of temperature on destabilizing stand growth, which may be mitigated by mixing species. We further confirmed asynchrony as the main driver of temporal stability in mixed stands, through both the additive effect and species interactions, which modify between-species asynchrony in mixtures in comparison to monocultures. Synthesis and applications. This study highlights the emergent properties associated with mixing two species, which result in resource efficient and temporally stable production systems. We reveal the negative impact of mean temperature on temporal stability of forest productivity and how the stabilizing effect of mixing two species can counterbalance this impact. The overyielding and temporal stability of growth addressed in this paper are essential for ecosystem services closely linked with the level and rhythm of forest growth. Our results underline that mixing two species can be a realistic and effective nature-based climate solution, which could contribute towards meeting EU climate target policies.Emerging stability of forest productivity by mixing two species buffers temperature destabilizing effectpublishedVersio

Emerging stability of forest productivity by mixing two species buffers temperature destabilizing effect

The increasing disturbances in monocultures around the world are testimony to their instability under global change. Many studies have claimed that temporal stability of productivity increases with species richness, although the ecological fundamentals have mainly been investigated through diversity experiments. To adequately manage forest ecosystems, it is necessary to have a comprehensive understanding of the effect of mixing species on the temporal stability of productivity and the way in which it is influenced by climate conditions across large geographical areas. Here, we used a unique dataset of 261 stands combining pure and two-species mixtures of four relevant tree species over a wide range of climate conditions in Europe to examine the effect of species mixing on the level and temporal stability of productivity. Structural equation modelling was employed to further explore the direct and indirect influence of climate, overyielding, species asynchrony and additive effect (i.e. temporal stability expected from the species growth in monospecific stands) on temporal stability in mixed forests. We showed that by adding only one tree species to monocultures, the level (overyielding: +6%) and stability (temporal stability: +12%) of stand growth increased significantly. We identified the key effect of temperature on destabilizing stand growth, which may be mitigated by mixing species. We further confirmed asynchrony as the main driver of temporal stability in mixed stands, through both the additive effect and species interactions, which modify between-species asynchrony in mixtures in comparison to monocultures. Synthesis and applications. This study highlights the emergent properties associated with mixing two species, which result in resource efficient and temporally stable production systems. We reveal the negative impact of mean temperature on temporal stability of forest productivity and how the stabilizing effect of mixing two species can counterbalance this impact. The overyielding and temporal stability of growth addressed in this paper are essential for ecosystem services closely linked with the level and rhythm of forest growth. Our results underline that mixing two species can be a realistic and effective nature-based climate solution, which could contribute towards meeting EU climate target policies

Commitment and coordination in boundedly rational interactions

Computational intelligence has become an integral part of human society. Algorithms recommend us news to read during breakfast, help us choose routes to reach workplaces, or suggest restaurants to dine at; but, in the end, it is us - humans - who have the final word in these interactions. And we choose as we please, driven by copious motives and goals. In many cases, we remain oblivious to the fact that our actions eventually shape the entire society, influencing other people's options and decisions. From this interdependence of our behavior stems the need for a theory of greater human interaction. And while traditional game theory offers an elegant body of work, its results are built upon an inherently flawed assumption - the absolute rationality of decision-makers, which, as many experiments in the past decades demonstrated, human psyche seems to show rather a lack of. This thesis attempts to venture beyond this assumption and provides a series of results characterizing the implications of deviating from full rationality. To this end, it adopts perhaps the most widely accepted model of limited cognitive abilities called the quantal response. Quantal response assumes humans act stochastically, choosing actions with higher utilities more frequently. To aid in human decision-making, we study domain-agnostic integration of quantal response into two celebrated interaction archetypes: the models of commitment and coordination. These archetypes have been successfully applied to many specific real-world scenarios, and computed strategies were shown to greatly improve in efficacy when behavioral models are incorporated. Yet, the methods developed for solving these scenarios are not transferable to general models of commitment and coordination, where the integration of bounded rationality has never been addressed until now. Commitment is an ability of a single leading agent to influence the course of play even before the interaction starts by publicly announcing a strategic behavior they will truthfully follow. In order for the commitment to be respected by other agents, the so-called leader has to serve a prominent role in society. The capacity to adopt this role is hence commonly reserved for established market leaders or large governmental institutions. Under mild assumptions, the commitment was shown to benefit the leader greatly, and carefully crafted strategies may hence assist in optimizing social good in the entire society. We show how to integrate quantal response into commitment models in order to account for human-type behavior, we identify the problem's computational complexity, and design algorithms computing commitment strategies with guaranteed convergence and bounded error. Coordination then further improves the agents' strategic capabilities by letting them act upon their interests not only based on the reasoning about the leader's or their other opponents' behavior, but also by conditioning their strategies on external private signals. The process of selecting and revealing the signals is traditionally entrusted to a mediator mechanism, referred to as the correlation device. By optimizing the device's public distribution over signals, coordination facilitates reaching socially desirable outcomes previously considered unattainable. We investigate how quantal response affects the topology of the solution space, how computationally difficult it is to compute the solutions, and design algorithms that traverse the solution space while optimizing the signaling structure. We evaluate the scalability and robustness of all the introduced methods on multiple domains characteristic to commitment or coordination scenarios. The reported results indicate that our methods are sufficiently precise and surpass the contemporary state-of-the-art non-convex optimization solvers by several orders of magnitude in terms of computation speed. We hope our efforts may expedite the adoption of game-theoretic methods for designing more efficient and egalitarian systems.Doctor of Philosoph

Generalization of the Polygon-Crossing Problem

We study the maximum possible number of intersections of a simple k{gon with a simple l{gon for k; l 3 odd. We generalize this problem to a larger class of objects. We prove that the number of intersections cannot exceed kl k (l 3)=2. This improves the best known upper bound and gives alternative solution of the case l = 5

MgxMn(1−x)(BH4)2 (x=0–0.8), a cation solid solution in a bimetallic borohydride

A solid solution of magnesium and manganese borohydrides was studied by in situ synchrotron radiation X-ray powder diffraction and infrared spectroscopy. A combination of thermogravimetry, mass and infrared spectroscopy, and atomic emission spectroscopy were applied to clarify the thermal gas desorption of pure Mn(BH4)2 and a solid solution of composition Mg0.5Mn0.5(BH4)2. MgxMn(1−x)(BH4)2 (x = 0–0.8) conserves the trigonal structure of Mn(BH4)2 at room temperature. Manganese is dissolved in the hexagonal structure of α-Mg(BH4)2, with the upper solubility limit not exceeding 10 mol.% at room temperature. There exists a two-phase region of trigonal and hexagonal borohydrides within the compositional rangex = 0.8–0.9 at room temperature. Infrared spectra show splitting of various vibrational modes, indicating the presence of two cations in the trigonal MgxMn(1−x)(BH4)2 solid solutions, as well as the appearance of a second phase, hexagonal α-Mg(BH4)2, at higher magnesium contents. All vibrational frequencies are shifted to higher values with increasing magnesium content. The decomposition temperature of the trigonal MgxMn(1−x)(BH4)2 (x = 0–0.8) does not vary significantly as a function of the magnesium content (433–453 K). The desorbed gas contains mostly hydrogen and 3–7.5 mol.% diborane B2H6, as determined from analyses of the Mn(BH4)2 and Mg0.5Mn0.5(BH4)2 samples. An eutectic relation between α-Mg(BH4)2 and LiBH4 is observed. The solid solution MgxMn(1−x)(BH4)2 is a promising material for hydrogen storage as it decomposes at a similar temperature to Mn(BH4)2, i.e. at a much lower temperature than pure Mg(BH4)2 without significantly losing hydrogen weight capacity thanks to substitution of Mn by Mg up to 80 mol.%. The questions of diborane release and reversibility remain to be addressed